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(57) ABSTRACT 

A method for digital watermarking and, in particular, for 
digital data hiding of significant amounts of data in images 
and video. The method employs a discrete wavelet transform 
for embedding gray scale images which can be as great as 
25% of the host image data. A simple control parameter is 
used that can be tailored to either hiding or watermarking 
purposes, and is robust to operations such as JPEG com- 
pression. The method also uses noisc-rcsiUcnl channel codes 
based on multidimensional lattices which can provide for 
embedding signature data such as gray-scale or color 
images. Furthermore, embedded image data can be recov- 
ered in the absence of the original host image by inserting 
the data into the host image in the DOT domain by encoding 
the signature OCT coefficients using a lattice coding scheme 
before embedding, checking each block of host OCT coef- 
ficients for its texture content, and appropriately inserting 
the signatured codes depending on a local texture measure. 
The method further provides for source coding the signature 
data by vector quantization, where the indices are embedded 
in the host by perturbing it using orthogonal transform 
domain vector pertuibations. The transform coefficients of 
the parent data are grouped into vectors, and the vectors are 
perturbed using noise-resilient channel codes derived from 
multidimensional lattices. The perturbations are constrained 
by a maximum allowable mean -squared error that can be 
introduced in the host. Also, speech can be hidden in video 
by wavelet transforming the host video frame by frame, and 
perturbing vectors of coefiBcients using lattice channel codes 
to represent hidden vector quantized speech. The embedded 
video is subjected to H.263 compression before retrieving 
the hidden speech. 

3 Claims, 18 Drawing Sheets 
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REFERENCE TO A MICROFICHE APPENDIX 
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NOTICE OF MATERIAL SUBJECT TO 
COPYRIGHT PROTECTION 

All of the material in this patent document is subject to 
copyright protection under the copyright laws of the United 
States and of other countries. The owner of the copyright 
rights has no objection to the facsimile reproduction by 
anyone of the patent document or the patent disclosure, as it 
appears in the United States Patent and Trademark Office file 
or records, but otherwise reserves all copyright rights what- 
soever. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

This invention pertains generally to encoding and decod- 
ing data, and more particularly to a method for embedding 
data in still images and video frames. 

2. Description of the Background Art 

As multimedia data becomes widespread, such as on the 
internet, there is a need to address issues related to the 
security and protection of such data, as well as to ensure 
copyright protection. Most multimedia data sources are 
readily accessible to, and downloadable by, all users of the 
internet. While access resu-iction can be provided using 
electronic keys, they do not offer protection against further 
(illegal) distribution of such data. 

Digital watermarking is one approach to managing this 
problem by encoding user or other copyright information 
directly in the data. The purpose of digital watermarking is 
not to restrict use of multimedia resources, but to resist 
attack from unauthorized users. 

While watermarking of image data could be visible, such 
as a background transparent signature, a visible watermark 
may not be acceptable to users in some contexts. Therefore, 
it is preferable to digitally watermark and image by invisibly 
hiding a signature information into the host image. The 
signature is then recovered using an appropriate decoding 
process. 

In order to be effective, an invisible watermark should be 
secure, reliable, and resistant to common signal processing 
operations and intentional attacks. Recovering the signature 
from the watermarked media could be used to identify the 
rightful owners and the intended recipients as well as to 
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authenticate the data. In this paper we are mainly interested 
in embedding data such that the signature is invisible in the 
host image. The challenge is to simultaneously ensure that 
the watermarked image be perceptually indistinguishable 

5 from the original, and that the signature be recoverable even 
when the watermarked image has been compressed or 
transformed by standard image processing operations. 

Research on digital watermarking can be categorized into 
two broad classes depending on the data embedding domain. 

10 One such class is based on embedding data in the spatial 
domain, while the other is based on injection in the fre- 
quency or transform domain. Most of the recent research on 
watermarking emphasizes the transform domain approach. 
Targeted applications include watermarking for copyright 

15 protection or authentication. Typically, the data used to 
represent the digital watermarks are a very small fraction of 
the host image data. Such signatures include, for example, 
pseudo-random numbers, trademark symbols and binary 
images. Spatial domain methods usually modify the least- 

2^ significant bits of the host image, and arc, in general, not 
robust to operations such as low-pass filtering. Much work 
has also been done in modifying the data in the transform 
domain. These include DCT domain techniques and wavelet 
transforms. 

While most of the contemporary research on watermark- 
ing concentrates on copyright protection in internet data 
distribution, a different kind of watermarking, commonly 
known as data hiding, is at present receiving considerable 
attention. Data hiding is a generalization of watermarking 
wherein perceptually invisible changes are made to the 
image pixels for embedding additional information in the 
data. Data hiding is intended to hide larger amounts of data 
into a host source, rather than just to check for authenticity 
and copyright information. In fact, the problem of water- 
marking or copyright protection is a special case of the 
generic problem of data hiding, where a small signature is 
embedded with greater robustness to noise. 

Data hiding provides a mechanism for embedding control, 
descriptive, or reference information in a given signal. For 
example, this information can be used for tracking the use of 
a particular video clip, e.g., for pay-per-use applications, 
including billing for commercials and video and audio 
broadcast. Data hiding could be quite challenging if one 

^2 considers embedding one image in another image. 

There has also been work on data hiding in color images. 
Once method is to use an amplitude modulation scheme 
wherein signature bits are multiply embedded by modifying 
pixel values in the blue channel. The blue channel is chosen 

50 as the human visual system is less sensitive to blue than 
other primary colors. Also, changes in regions of high 
frequencies and high luminance are less perceptible, and 
thus are favorable locations for data embedding. Robustness 
is achieved by embedding the signature several times at 

55 many different locations in the image. Another approach is 
use the S-CIELAB, a well-known standard for measuring 
color reproduction errors. In that approach, amplitude - 
modulated sinusoidal signals are embedded into the yellow- 
blue color band of an opponent-color representation scheme. 

60 It will also be appreciated that, in perceptual data hiding, 
one is interested in embedding and recovering high quality 
multimedia data, such as images, video and audio. The host 
multimedia data itself could be subject to signal processing 
operations, typically compression. Depending on the end 

65 user application, both lossy and lossless data embedding is 
of interest. Like in digital watermarking, two scenarios are 
possible. One is that the original host into which the data is 
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embedded is available. Alternatively, the original host infer- In accordance with a further aspect of the invention, color 

mation may not available. This is a much more difficult signature images are fused in larger color images using 

problem. wavelet transforms and lattice structures. We use the YUV 

Data hiding can also be used for transmitting different color space for representing color. The Y component is the 
kmds of information securely over an existing channel s luminance part of the signal, and U and V represent the 

dedicated for transmitting something else, such as transmit- chrominance components. Adopting the YUV color space 

ling hidden speech over a channel meant for transmitting facilitates a simple extension from images to digital video 

H.263 video, as in this work. Since a substantial amount has such as those in the MPEG format. The U. V components arc 

already been invested in the development of the software down -sampled by a factor of two. In this method, the host 
and hardware infrastructure for standard-based data lo and signature images are first wavelet transfonned used the 

transmission, it makes monetary sense to try lo use the same discrete Haar wavelet transform. The wavelet coeflScients 

for transmission of secure or non-standard data. are then encoded using channel codes derived from a finite 

BRIEF SUMMARY OF THE INVENTION subset of the lattice structure, which consists of all integer 

. . N-tupIes with constraints. As the quantity of embedded data 
In general terms, the present invention pertams to a data ^5 increases, higher order shells of the lattice structure are 

embedding scheme that is suitable for both watermarking included in the channel code to accommodate them, 

and image data hiding While watermarking requires robust- ,^ accordance with a further aspect of the invention, a 

ness to image manipulation, data hiding requires that there .-ij • ujj- .^jr j.lj- l 

1**1 • ui J- * • L r- «7i- i L spatial domain embedding method for data hiding speech 

is very little visible distortion in the host image. While much a a ■ ™ a -a - * ^ u ^ u% 

f , J • * J * Tu * • 11 video m compressed video is presented based on bit 

01 the previous work used signature data that is a small 1 * o 1 j - * . • . 

. ■ f*u u « • -1 * *u » • *- ^ replacement. Spatial domam strategies are quite sensitive to 

fraction of the host image data, the present mvenUon can . ^ ^ u j • 1 j . 

. J, , ^. * 1 . . . transformations on the embedded signal. Compared to con- 
easily handle gray-scale images that could be as much as 1 . l • 41. • l j • c .1 
oc/ff <^ *u u . • ventional tcchmqucs, the invention can embed significantly 
25% of the host image. , . / • * j . ■ . *u u . * ^r/w c 
. * ^ . . . . larger amount of signature data into the host — ^up to 25% of 
In accordance with one aspect of the invenUon, m recov- ^i^, ^^^^ ^^j^^ ^^^^^ perceptual distortion, 
enng the signature image, it is assumed that the original host . * r *u • ■ * i_ j - n 

• -1 ui m. • *■ J- * -i- . • . An object of the mvention is to embed a significant 

image is available. The mvention distributes the signature A j,*, • ;„,„^, ^^au. „;a.^ 

• c • *u J- . 1 * . r ^^TTin amount 01 data m images and/or video, 

information in the discrete wavelet transform (DWT) * . i- l - • • . , ^ . , .. 

domain of the host image. Spatial distribution of the DWT P^^^^^ "^^^^^^^^S 

coefi&cients helps to recover the signature even when the Quahty control m data transmission (e.g., self-enhancing 
images are compressed using JPEG lossy compression. In 30 "^^^es), cmbeddmg control mformation in audio/visual bit 

some of the recent work on using wavelets for digital watermarking, 

watermarking, the signamres were encoded in all DWT ^^tihcr objects and advantages of the mvenUon will be 

bands. Such an embedding is sensitive to operations that ^^^^^ht out in the following portions of the specification, 

change the high frequency content without degrading the wherein the detailed description is for the purpose of fuUy 
image quality significantly. Examples include low pass 35 d^sclosmg preferred embodiments of the invention without 

filtering for image enhancement and JPEG lossy compres- P^^^'^S limitations thereon. 

sion. In contrast, the present invention focuses on hiding the BRIEF DESCRIPTION OF THE DRAWINGS 

signature mostly in the low frequency DWT bands, and -^^^^^^^ ^^^^ .inderstood by reference 

stable reconstrucuon can be obtained even when the images ^^^^ ^^^^ ^^^^^ illustrative purposes 



are transformed, quantized (as in JPEG), or otherwise modi- 40 
fied by enhancement or low pass filtering operations. 



only: 

- , . ... FIG. 1 is a block diagram of a method for embedding 

In accordance with another aspect of the invention, it is 1 *„ ■ j- * 1 * * f 

1 J *t- L * • - M 1.1 1^ • gray-scale images using a discrete wavelet transform 

also assumed that the host unage is available. The invention ^^/^rA;r^^ thZ ;r,„<>«H-^r, tWo ..;nr«.H.« 

L . J . L-j* . L . . . 1 1 accoramg to the invention, where the signature image is 

provides a robust data hiding technique using channel codes j k» ™« iu u * - a 

j.,^ L . r 1 r 1, assumed to be one quarter the size 01 the host image, and 

derived from a finite subset of general n-dimensional la - 45 ^^ere there is shown an expansion of a single signature 

tices. In parucular we use the latUce, which consists of all ^oefBcient lo a 2x2 block of coefficients for embedding in 

integer n-tuples with an even sum. As the quantity of ^^^^ imaee 

embedded data increases, higher order shells of the lattice -i • ^ ^ * ^ • 

J4«i. FIG. 2 IS a graph showmg the presence of a signature in 

are included m the channel code to accommodate them. , j. l .ll.- t * 

Using this approach, a gray-scale image of as much as half 50 ' '""^P;"^'^ ^"^'^^ ^ " ^^'^^ 

thesizeof thehostimagecanbeembeddedbyperturbingthe /he signature is a Uger miage. 

host wavelet coefficients ^ ^ ^ S^^P^ showmg the presence of a signature in 

Theembeddingandextractingofthedigital watermarking ^ ^^^^^ compressed image where the host is a cityscape 

system are similar lo the encoder and deader of the digital ^"^^^^e and the signature is an airplane image, 

communication system. Similar lo the communication chan- 55 f^p- ^ ^'^^'^ P^^'*''^ P'^^y perturbations 

nel noise, the watermarked image might undergo undesir- » ^""^^ ^^^^^ ^^^^''^ P^^"*^ shown in n-dimensional 

able transformations: for example, intentional manipulations space. 

to remove or degrade the quality of the watermarking: or diagram showing possible noisy vector posi- 

typical signal processing operations such as compression ^^o"^ ^" original perturbed vector s,. after transformation 

that may affect the watermark. We use a wavclct-bascd 60 where all points are shown in n-dimensional space, 

compression scheme, and the JPEG compression scheme for PIG. 6 is a block diagram showing a data embedding and 

the manipulation of the watermarked image before attempt- extraction method using multidimensional lattices according 

ing retrieval. Our experimental results indicate that there are to the present invention. 

no visible distortions in the watermarked image, and the FIG. 7 is a block diagram of the encoder block shown in 

recovered signature is similar to the original signature even 65 FIG. 6 for encoding gray scale images, 

after 75% wavelet compression and 85% JPEG lossy com- FIG. 8 is a block diagram of the decoder block shown in 

pression. FIG. 6. 
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FIG. 9 is a diagram showing the decision boundary within 
each of a plurality of shell perturbed lattice points. 

FIG. 10 is a graph showing the presence of a hat-girl 
signature in JPEG lossy compressed images for a=10, p=32; 
a-15, p-32; a-10, p=144; and a-lS, P-144. 

FIG. 11 is a block diagram of an alternative embodiment 
of the encoder shown in FIG. 7 for embedding color images. 

FIG. 12 is a diagram showing determination of the closest 
vector from the observed vector within each of a plurality of 
shell perturbed lattice points. 

FIG. 13 is a graph showing similarity results for color data 
embedding. 

FIG. 14 is a graph showing PSNR results for color data 
embedding. 

FIG. 15 is a block diagram of a method for data embed- 
ding for reconstruction without the host image according to 
the present invention where data is embedded in the block 
DCT domain, signature DCT coeflScicnts are quantized, 
coded using lattice codes, and adaptively embedded into the 
host DCT coefiScients using texture masking. 

FIG. 16 is a diagram showing a sample signature quan- 
tization matrix for an 8x8 DCT coefiBcient block, requiring 
112 host image coefficients to encode. 

FIG. 17 is a diagram showing partitioning of the DCT 
block of FIG. 16 for signal insertion (shaded regions) where 
18 coefficients are used in each block. 

FIG. 18 is a diagram showing a sample signature quan- 
tization matrix requiring 192 host coefficients. 

FIG. 19 is a diagram showing partitioning of the DCT 
block of FIG. 18 where the host coefficients are distributed 
over 16 blocks, 12 coefficients per block, as shown by the 
shaded regions. 

FIG. 20 is a block diagram of the encoder block shown in 
FIG. 15. 

FIG. 21 is a graph showing the PSNR of embedded and 
recovered host images as a function of JPEG compression 
ratio with a scale factor of 5, wherein the solid lines 
represent 6% embedding using the quantization matrices of 
FIG. 18 and FIG. 19, and wherein the dashed lines shown the 
results at 25% embedding using the quantization matrices of 
FIG. 16 and FIG. 17. 

FIG. 22 is a graph showing the PSNR of the recovered 
signature image for the images of FIG. 21 as a function of 
JPEG compression ratio with a scale factor of 5, wherein the 
solid lines represent 6% embedding using the quantization 
matrices of FIG. 18 and FIG. 19, and wherein the dashed 
lines shown the results at 25% embedding using the quan- 
tization matrices of FIG. 16 and FIG. 17. 

FIG. 23 is a schematic showing the data hiding and 
watermarking problem. 

FIG. 24 is a diagram showing the principle of data 
embedding in relation to FIG. 23. 

FIG. 25 is a diagram showing the principle of data 
extraction in relation to FIG. 24. 

FIG. 26 is a diagram showing a two stage wavelet 
decomposition of each frame for recovery from a video host 
without the original video, where the data is hidden in the 
shaded LL-HH subband after zeroing. 

FIG. 27 is a schematic showing a method for data 
encoding in video according to the resent invention using the 
zeroed LL-HH subband. 

FIG. 28 is a schematic showing a method for data 
decoding in video according to the present invention using 
the zeroed LL-HH subband. 
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FIG. 29 is a graph showing the SNR of extracted hidden 
male speech vs. bit rate for an H.263 compressed "News" bit 
stream at 15 frames/s for D4, Eg and Ajg lattice implemen- 
tations of the data hiding and recovery method depicted in 
5 FIG. 27 and FIG, 28. 

FIG. 30 is a graph showing the SNR of extracted hidden 
female speech vs. bit rate for an H.263 compressed "grand- 
mother" bit stream at 7.5 frames/s for Eg, lattice 
implementations of the data hiding and recovery method 
30 depicted in FIG. 27 and FIG. 28. 

DETAILED DESCRIPTION OF THE 
INVENTION 

Referring more specifically to the drawings, for illustra- 
tive purposes the present invention is described with refer- 
ence to FIG. 1 through FIG. 30. It will be appreciated that 
the invention may vary as to configuration and methodology 
without departing from the basic concepts as disclosed 
^ herein. 

1, Data Embedding 

A watermark should be robust to typical image processing 
operations, including lossy compression. Compression 
techniques, such as JPEG, typically affect the high fre- 
quency components. This is also true with most perceptual 
coding techniques. For these reasons, a digital signature 
should be placed in perceptually salient regions in the data. 
For techniques based on frequency domain modifications, 
this implies embedding the signahire in mostly low fre- 
3Q quency components. Inserting a signature in low frequency 
components creates problems if one is interested in invisible 
watermarks. This is particularly true in data hiding applica- 
tions where the data to be hidden could be a significant 
percentage of the original data. 
35 To address this problem, the present invention uses a 
wavelet transform to embed signature information in differ- 
ent frequency bands. For experimental purposes we used the 
discrete Haar wavelet basis; however, those skilled in the art 
will appreciate that extending the invention to another 
wavelet basis is reasonably straightforward. Both the signa- 
ture data, which in our case is another image, and the host 
image data, are decomposed using the discrete Haar wavelet 
transform (DHWT). 

In the following discussion it is assumed that the signature 
45 image is one quarter the size of the host image, and both 
images are gray scale, one byte per pixel. Embedding occurs 
in the wavelet transform domain as the wavelet coefficients 
are combined to create a watermarked image. It is assumed 
that the host image is available for signature image recovery. 
A schematic of this approach is shown in FIG. 1. 

The basic steps in embedding the signature coefficients 
into the host image coefficients are: 

1 . Decompose by one level the host and signature images 
using the DHWT. This results in four bands, which arc 

55 usually referred to as the LL, LH, HL, and the HH bands as 
shown in block 10. 

2. Each signature image coefficient is expanded into a 2x2 
block as follows: 

(a) Each coefficient value is linearly scaled to a 24 bit 
60 representation as shown in block 12. 

(b) Let A, B, C represent, respectively, the most signifi- 
cant byte, the middle byte, and the least significant byte 
in a 24 bit representation. Three 24-bit numbers, A', B', 
C, are generated with their most significant bytes set to 

65 A, B, and C, respectively, and with their two least 
significant bytes set to zero as shown in block 14. Then 
a 2x2 expanded block is formed as shown in block 16. 
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3. The host image coefiBcients are also linearly scaled In checking for the presence of a signature, the quality of 
within each band to a 24 bit representation. The minimum the reconstruction of signature itself is not an issue. A binary 
and maximum values in each band will be used in the decision for the presence or absence of a signature need to 
inverse transformation described below. be made. We used a measure of "similarity" S to compute 

4. The scaled host image coefficients are now added to the 5 the cross correlation between the recovered signature s*(m, 
expanded signature transform to form a new fused trans- n) and the original signature s(m, n) in the wavelet transform 
form. Let h(m, n) be the (m, n)''' wavelet coefiGcient of the domain. This similarity is defined as: 

host image, and let s(m, n) be the (m, n)'* signature 

coefficient after forming the expanded blocks as described in y ^* (2) 

Step 2. Note that after expansion, each of the bands in the ' 

signature wavelet transform is of the same dimension as the ^~ Z(s'[m,n))^ 
host image bands. The fused (m, n)''' coefiBdent is then 
computed as: 

Hn,n)^in,n'Mn,n) (1) ^otc that the similarity conaputed as above does not guar- 

15 antee that the maximum value is 1.0. Graphs of this suni- 

where the scale factor a determines the relative percentage larfty for varying JPEG compression and for different scale 

of the host and signature image componenU in the new factors for two different examples are shown in FIG. 2 and 

^°^^Se- FIG. 3. In both graphs, the scale factors were ao5, a=7, a^9 

5. The fused transform coefiBcients in each band are scaled and a«.ll . As can be seen from those graphs, it is easy to find 
back to the levels of the host image transform coefiBcients 20 a threshold for signature detection between unwatermarked 
using the minimum and maximum coefiBcient values in Step and watermarked images, 

3* The foregoing method can be used for both digital water- 

6. An inverse transform is now computed to give the marking related applications as well as for data hiding 
watermarked image. purposes. The scale factor in Equation (1) controls the 

EXAMPLE 1 ^ relative amount of host and signature image data in the 

, . 1, r .... 1^0 1 embedded image. A larger scale factor can be used for data 

we presem nere results or emoeaamg iz«xiz« gray scale ^^^^^ 

it is desirable to maintain the oerceotual 

(one byte per pixel) signamre images m a 256x256 Una quality of the embedded image. A lower scale factor is better 

miage. TWo images, one a hat girl picture and a p.aure of ^^^/^^ watermarking where robustness to typical image 

a tiger, were used as signature unages id the following ^„ _ • . a a r: • * i n j 

f r- T J 30 processing operations IS needed. Expenmcntai results dem- 

experiments. Scale factors of a=5, a=7, and aall were ♦ ♦ .u * j i-* • *, ^ 

' ' onstrate that good quality signature recovery and authenti- 

, , . . . , ^ . . . cation is possible when the images are quantized and JPEG 

We noted that the higher the scale factor, the better the compressed by as much as 90%. 

quality of the embedded image (i.e., less distortion due to ^ appreciated that, even though the Haar wavelet 

embedding) Even if the signature image has much texture 35 ^asis was used in the experiments, the method can be easily 

information like a tiger picture, the embedded miage cannot .^^p^^^ ^^^^^^^ transforms and for more than one 

be visuaUy distmguished from the onginal host rniage. i^^^i decomposition. It might be worth exploring the use 

Two sets of experiments were conducted. In the first, for of other basis functions depending on the characteristics of 

data hiding applications, results of signature image reoon- xhe host and signature images. In some cases, particularly 

struction from JPEG lossy compressed images for varying 40 when the host image background lacks texnire whereas the 

scale factors were determined. In die second, for watermark- signature image has lot of texture, one can see a noisy 

ing applications, we determined the results of signature background in the embedded image, 

detection from these lossy compressed images. in digital watermarking, the signamres are usually of 

For data hiding purposes it is reasonable to choose a larger much smaller dimensions (in terms of number of bytes 

scale factor in Equation (1) because we are not too con- 45 needed) compared to the host image. Since the method 

cemed about degradation due to image processing opera- described above can manage a significantly larger number of 

tions. In hiding one image in another, it is more important to signature data, it is possible to distribute the signature 

ensure that the quality of the watermarked image is as close spatially as well, thus making watermarking robust to opera- 

to the original as possible, with very lit tie visual distortion. tions such as image cropping. 

Almost perfect reconstruction is possible when there is no so 2. Multidimensional Lattice Channel Cbde 

further image processing of the watermarked images. 2.I Methodology 

On the other hand, for copyright protection and authen- If the original host image is available, the operations of 

tication purposes it is important that the watermarked data injection and retrieval are, in fact, very similar to the 

images are robust to typical image processing operations. In channel coding and decoding operations in a typical digital 

such cases it is reasonable to assume that the signatures ss communication system. Channel coding refers to the gamut 

consume significanUy fewer bytes than the host image and of signal processing done before transmission of data over a 

as such can be spatially distributed. In our experiments we noisy channel. In watermarking in the transform domain, the 

used lossy JPEG compression where the signatures are the original host data is transformed, and the transformed coef- 

scale images, and it is reasonable to expect that one can flcients are perturbed by a small amount in one of several 

obtain much better results if the signatures are binary images 60 possible ways in order to represent the signature data. When 

of much lower dimensions. Lower values for the scale factor the watermarked image is compressed or modified by other 

in Equation (1) should be used when it is likely that the image processing operations, noise is added to the already 

images undergo significant distortion. We recovered signa- perturbed coefficients. The retrieval operation subtracts the 

tures for JPEG compression of 93% for scale factors of a-3 received coefficients from the original ones to obtain the 

to a-U. As expected, images embedded with larger scale 65 noisy perturbations. The true perturbations that represent the 

factor resulted in poor reconstruction for the same compres- injected data are then estimated frt)m the noisy data as best 

sion factor. as possible. 
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In the present invention, we have adopted a vector-based 
approach to hidden data injection. We group N transform 
coefficients to form an N-dimensional vector, and modify it 
by codes that represent the data to be embedded. The 
motivation for tising vector perturbations as opposed to S 
scalar perturbations follows from the realization that higher 
dimensional constellations usually result in lower probabil- 
ity of error for the same rate of data injection and the same 
noise statistics. 

FIG. 4 and FIG. 5 show the basic concept of the pcrtur- lO 
bation vector in the host N-dimensional vector space. In both 
figures, "x" represents a host vector in an N-dimensional 
space. To embed data from an p-ary source with symbols 
{Sj, Sj, . . . , Sp}, we perturb the original vector so that the 
perturbation coincides with one of p corresponding channel 15 
codes. The perturbed vector is denoted by one of the "o"s in 
the figures, depending on the particular source symbol it 
represents. After the watermarked image has undergone 
compression or other transformations, a perturbed vector 
representing, for example symbol s^ in the diagram, may be 20 
received as a noisy vector in FIG. 5. It is then an 
estimation problem to extract the transmitted symbol from 
the vector received. Assuming an additive Gaussian noise 
model, the received vector is decoded as representing the 
symbol whose channel code it is closest to in Euclidean 25 
distance. 

Codes derived as subsets of multidimensional lattices 
have been shown to be very efficient for channel coding. In 
the following, we describe the general concept of lattices, 
and in particular, the D4 lattice that was used in our data 30 
embedding algorithm. 

2.2 Lattice Structures 

The Voronoi regions of various n-dimensional lattices can 
be used to construct n-dimensional quantizer cells for uni- 
formly distributed inputs. It is known that some of these 35 
lattices produce very good channel codes, and yield high 
values of nominal coding gain. That is, for the same power 
constraint on the channel, the channel codes are maximally 
separated from each other so that they are most robust to 
noise. The lattices considered here are the root lattices and 4o 
their duals, namely A„, A*„, D„, D*„, Eg, Eg, etc. If a^, . . . , 
a„ are n linearly independent vectors in an m-dimensional 
Euclidean space with m in, the set of all vectors 



(3) 



45 



where Uj, . . . , u„ are arbitrary integers, constitute an 
n-dimensional root lattice A„. F\irther, if A is a lattice in gi", 
the dual lattice A* consists of all points x in the span of A 
such that x-yez for all yeA. Some common lattices and 
definitions are presented below, 50 

For nil, A„ is the n-dimensional lattice consisting of the 
points (Xq, Xi, . . . , xJ in Z"*^ with £x,=0. 

For ni2, D„ consists of the points (xj, X2, . . . , x„) in Z" 
with Zx,- even. In other words, if wc color the integer lattice 
points alternately red and blue in a checkerboard coloring, 55 
D„ consists of the red points. In 4 dimensions, the D4 lattice 
is known to yield the best coding gain. 

The Eg, Eg and A^g lattices give very good channel coding 
gains in 6, 8, and 16 dimensions respectively. The Eg lattice 
is derived from the Dq lattice, and is defined as the union of 60 
Dg and the cosset 

In other words, E^ consists of the points (x^, . . . , Xg) with 
x, eZ and 2x,- even, together with the points (yi , . . . , y^ with 65 
y, eZ+V^ and Sy^ even. Eg is a subspace of dimension 6 in Eg, 
consisting of the points (uq, u^, . , . , u,J with ug-u^— Ug. 



For a n-dimeosional lattice A, the \bronoi region around 
any lattice point is the set of points in g?" closest to the 
lattice point. Therefore, the Voronoi region V(0) around the 
origin is given as: 

VtOXxcgflllxllSllx-M |[(for all nonzero u£fl)} (4) 

23 Description of the Lattice 

It is known that some lattices produce very good spherical 
codes for channel coding. That is, for the same constraint on 
deviation from the true coefficient values, the channel codes 
are maximally separated from each other so that they are 
most robust to noise. 

In general, the D4 root lattice produces the best channel 
code in 4 dimensions. It is known that for small noise, this 
lattice gives a nominal channel coding gain of 1.414 over 
binary encoding. As mentioned earlier, the lattice consists of 
the points (x^, . . . , X4) having integer coordinates with an 
even sum. 

As in all lattices, the lattice points of the D4 lattice fall on 
concentric sheUs of increasing distance from the all zero 
vector. For example, the 24 lattice points given by all 
permutations of (±1,±1, 0, 0) lie on the first shell of the 
lattice at a distance from the center. The second shell at 
distance ^ from the center contains 24 lattice points again, 
8 of which are of type (±2 , 0, 0, 0), and 16 are of type (±1, 
±1, ±1, ±1). Table 1 shows the shell number, the squared 
norm, the lattice point types, and the number of lattice points 
for the first few shells of the D4 lattice. The superscript "p" 
after the points in the table denote "all permutations of the 
elements constituting it. By choosing appropriate subsets of 
points from the lattice the rate for data embedding can be 
varied. 

3. Data Hiding in Images 
3.1 Embedding Procedure 

It is well known that embedding in the low-frequency 
bands is more robust to manipulations such as enhancement 
and image compression. However, changes made to the low 
frequency components may result in visible artifacts. Modi- 
fying the data in a multiresolution framework, such as a 
wavelet transform, will provide good quality embedding 
with little perceptual distortion. 

The schematic diagram 20 in RG. 6 shows our water- 
marking procedure using multidimensional lattice chaimel 
codes. The coefficient vectors perturbed in our implemen- 
tations arc of dimension 4, and the channel code used to 
embed the data is a subset of the D4 lattice. As the quantity 
of embedded data increases, higher order shells of the 
embedding lattice are included in the channel code to 
accommodate them. In this algorithm, a gray-scale image of 
as much as half the size of the host image is hidden by vector 
based perturbations. 

A single level of the discrete wavelet transform (DWT^ 
decomposition of both the host and the signature image is 
made before data embedding. A detailed diagram of the 
encoder block 22 from FIG. 6 is shown in FIG. 7. Each 
coefficient of the signature image is quantized into p levels. 
In order to embed the quantized coefficient information, a set 
of n coefficients (n=4 in the case of D4 lattice embedding) in 
the host image is grouped to form an n-dimensional vector, 
and the vector is then perturbed according to a p-ary channel 
code consisting of a subset of an n-dimensional lattice scaled 
by a factor a. If v represents a vector of host DWT 
coefficients after grouping, and the index of the quantized 
signature coefficient is i, then the perturbed vector is given 
by: 



(S) 
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where C(sJ tepiesents the channel code (subset of the wavelet-based compression or JPEG coding or other trans- 

• 11 *4- \ J- * *u LI ,t,^ formations like enhancement, if the embedded vector is 

n-dimcnsional lattice) corresponding to the symbol S;, where ^ , • i . j . • i . ^ 

0 / ft, / i» strongly manipulated, to say, noisy vector r*/ * , located 

c i. uil J • * • u jj J • * *L outside of the decision boimdary, the symbol detected will 

Eachsubbandof the signature image is embedded into the .... • ' , _^ j i j .u • • 

corresponding subband of the host. That is, each coefBcient ' P/"^^^ t*'"^ J° "^'^ 

in the LL biad of the signature image is hidden in four erroneous detection, he algorithm can expand the 

Fiujf*uu* j Tn. 1 decision boundary by usme a larger scale factor. 

coefBcients m the LL band of the host, and so on. The scale ^^^^ ... f . ^ . *• j • 

factor chosen for embedding in the higher bands is less than recovered signature >mage js lunited m 

the scale factor chosen for the LL b«id, by some constant ''"'''"y "'e quanUzaUon before embeddmg the simdanty 

factors. However, we wiU refer to the scale factor chosen for '° ^ '° ^quaUon (2) can be used to distmguish 

the LL band as a between watermarked and unwatermarked images. Here 

Various subsets of the 4-dimensional D, lattice chosen for ^F"' °) ^'fx^f ^ quantized signature coefficients and s 

1 c *• *• 1 I o *L . (m, n) stands for the recovered signature coefficients after 
various values of source quantization levels p, that were i • 

used in the experiments, are shown in Table 2. A high value Tf *thm 

of 6 quantizes the signature finely, but a must now be higher * * . - • i ... i_ j l t 

* .L . .u u iTi-. r • a: • .1 1 Tn.- - One of thc motivatious for usmg lattice based channel 

too so that the probability of erroris sufficiently low. This in , . . , ^ . ..... rr . j 

turn degrades ttie transparency of the watermarked image. ^'•^^^ implementations is the existence of fast encod- 

He choice of the parameters a and p determines the decoding algonthnK. We present a fast encoding 

trade-off between the transparency and the quaUty of the "Jg^th™ D- ^^^<=^ 'hat ts used to extract the hidden 
bidden data »- ^ / 20 symbols from the noisy vectors received, if the number of 

^ ... • u* * *' I 4 ' I channel symbols 6 is sufficiendy large. 

For secunty in copy nght protection, we can select special , r ^ _.• 1 * r^i. 1 . 

regions in the transform dotiain to embed data, or randomly "^Ig""""" P"""' °^ ^^^^J^ 

group the coefBcients to form a vector using a private key. an arbitrary scaled noisy pertuibation received x=(l/a) e e 

Noise-like pseudo-random sequences can be used for ran- js particularly simple. Note that all points of D„ are 

dom grouping. It is to be noted, however, that in general, the deluded in the n-dimensional cubic integer lattice I". For a 

less the quantity of data hidden, the more secure it can be ^^^j^^ ^^^^^^ f(x)«closest integer to x. We 



define f(x) and the function w(x) which assigns the wrong 



made. 

3.2 Extracting Data 3,2,1 Determining the Closest Point iT^aon'asTollows: 

A watermarked image may be subject to lossy compres- ^^^^ 

sion or other simple image processmg operations sudi as v / » v / 

enhancement. Under the assumption that the resulUng per- 0<m^x^m+V4, then f(x)-m, else w(x)-m+l, 

turbations in the wavelet transform domain can be modeled If 0<m+V^<x<m+l, then f(x)-m+l, else w(x)-m, 

by additive Gaussian noise, a nearest-neighbor search with If -m-V^^x^-m<0, then f(x)=*-m, else w(x)=-m-l, 

the Euclidean distance measure is needed to recover the if -m-l^x^-m-V^, then f(x)«-m-l, else w(x)=-m. 
embedded symbols. FIG. 8 provides a diagram of the ^ We can also write x«f(x)+6(x), so that |6(x)|^V^ is the 

decoder block 24 fi^om FIG. 6 to show the details of symbol distance from x to the nearest integer. Then, if x={xi, Xj, . 

recovery and signature exU-action. . . ^ xj, vector f(x) is defined by 

Recovering the hidden data starts with the same DWT of 
the received watermarked image that was used to embed the 

data. The true host image coefficients (known to the Aj=^)-{/(^l)>/(-*2)> • * • >/(^a)» ■ • • >AO} (7) 

retriever) are then subtracted from thc coefficients of thc g^^^^ defined by 
received image to obtain the noisy perturbations. Note that 

these perturbations recovered can be "noisy" , because of 8ix)~{f(xi), A^t)* ■ ■ • > Mxi^, Kx„)} (8) 

various possible transformations of the watermarked data. ^^^^^ ^ ^ component with the largest error distance. The 

These coefficients are now grouped into groups of n in the ^^^^^^^ ^ ^ ^^^^^ ^^^^^^^ chosen as 

same manner as they were grouped dunng en«)ding whichever of f(x) and g(x) has an even sum of components, 

(possibly using the private key) to obtain a vector e , and If x is equidistant from two or more points of thc lattice, we 

1 J u r 4 1 ; Tu u* 4 i / ~* • choose the nearest point as the one having the smallest norm, 

then scaled by the factor 1/a. Thc resulting vector 1/a- e is ^ 

then nearest-neighbor encoded to find the index i of the EXAMPLE 2 

channel code nearest to it in Euclidean distance. In „. . , , ., . . ^ 

particular, we find an index i such that: " ^56x256 gray scale Lena image as the host, and 

two signature images, a hat-girl image and a tiger image, 

uzi. ^ V u « ni both of which were 128x128 gray scale. A 1-stage discrete 

the decoder. 

where the C(sj 's refer to the S code-vectors m the channel „, • .u.. i ai^u^iu. u^a ... tu 

1 ii 1 t ... -1. We exammed the Lena image digitally watermarked with 

codebook. For lalUce based channel codes, Ihis |s equivalent j^^^.^j . ^^^^^ ^^^^^^ ^^^.^^^ 

to finding the lattice pomt in whose Voronoi region the q„,„ti2fti„„ ^J^^ p_ ^th^u, compression. Note that 

vector 1/a* e lies. From the index i, the quantized DWT 60 the scale factor a controls the relative weight of host and 

coefficient can be obtained. signature image contributions to the fused image. As a 

To present an example, by means of the diagram in FIG. increases, the quality of the watermarked image degrades. 

9, let us say that a perturbed vector corresponding to a For example, we could see artifacts in the background for 

channel code s, was received as a noisy vector r,- As long a«>20. We found that a=10 appears to be a reasonable value 
as it is inside the decision boundary of the original perturbed 65 in terms of the trade-ofif between quality of the watermarked 

vector s^, we can receive the data perfectly. However, after image and robustness to signature recovery under image 

the general image compression schemes, for example, compression. 
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We also examined the signature images recovered from triangular decision boundary shown, will be correctly 

the waterinarked image after 0%, 65%, 75% and 85% JPEG estimated. Obviously the scale factor a controls the extent of 

compression. In general, most of the recovered signature the regions around each s^. A large scale factor can tolerate 

images were of very high quality for 85% JPEG a large perturbation at the expense of a degradation in the 
compression, when the scale factor a is in the range 10-15. s watermarked image quality. 

The quality of the recovered signature with a large scale j^^ principal difference for data hiding in color images is 

factor a is obviously much better than those with a srnaller i^al color signature images are fused in larger color images 
a lh6_ number ot quantizer levels p, on the other harid, • ^^^^y^^ transforms and lattice structures. We use the 

determines the coarseness of quantization and therefore the viiv ^^.i^,. o«-...^ tk- v - ♦ 

quality of the signature image hidden in the host. YUV color space for representing color The Y component 
in u .u -t •* u * *u ■ • I J 10 IS the lummance part of the Signal, and U and V represent the 

FIG. 10 shows the similanty between the origmal and the „™ . aaTI' *u vtm/ i 

, . ■ chrominance components. Adopting the YUV color space 

recovered signature, when the hat-mrl imaee is embedded r -i * * i . • r • . j- i -j 

• , ,u 1 • ^ ^ ^ "<^^ B^'^ ^^<^& ^ ^i^^^y^^^y^ facihtates a simple extension from images to digital video 

mto toe Una image^ Note toat good avtoenUcaUon is ^^^^ ^ ^ j^pg^ ^ ^ components are 

po^rfjle for up to 85% JPEG lossy compression. down-sampled by a factor of two. In thfe method, the host 

/i^ can be seen from the foregoing, the mvcntion provides and signature images are first wavelet transformed used the 

for highly effective date embeddmg using toe latUce in discrete Haar wavelet transform. The wavelet coeflScients 

toe pWT domain. The method presented provides a frame- „e toen encoded using channel codes derived from a finite 

work for a more structured digital watermarking scheme, ^^bset of toe latUce structure, which consists of all integer 
aimed at embeddmg large amounts of data into a host. The constraints. As the quantity of embedded data 

quality of the recovered signature under significant image ^ increases, higher order shells of the lattice structure are 

transformations can be improved by using higher dimen- included in toe channel code to accommodate tocm. 
sional lattice structures uke the Eg or the lattice. Further, 

by proper indexing of the scalar codebook used for the EXAMPLE 3 

wavelet coefficients of the signature image, the recovered Color images were represented in the YUV color space, 
signature quality can be substantially improved for the same 35 We used a 256x256 color hose image and a 128x128 gray 

scale factor of embedding and for the same number of levels scale signature image. The signature was injected into the Y 

for quantization. More sophisticated schemes for error component of the transform coefficients of the host image, 

resilience, such as trcllis-codcd modulation, can also be prom observing an 81% JPEG compressed watermarked 

image using 32 channel codes and the same compressed 
4. Color Image Embedding Using Mulddimensional Lattice 30 image using 144 channel codes, we found that there were no 

Structures visible distortions in the watermarked images. Additionally, 

It IS known diat the human visual system is not very from observing the recovered signatures for the two quan- 

sensitive to changes in the higher frequency spectrum, and tization levels, we found the reconstructed images to be of 

as such many of the lossy compression techniques rely 00 very good quality for authentication purposes, 
saving bits needed to represent the information in these 35 We also examined an example of a color signature embed- 
higher frequencies. For this reason it is important that the ^h^ ^^^^^^ ^^^^^^^^ ^^^^ embedded in the Y 

signature data be embedded m the lower frequency compo- component of the host data in order not to distort the color 

nents of the host data. in the watermarked image. For this reason, the size of the 

Tlie schematic 30 in RG. 11 shows our color image signature image was less than that for a gray scale embed- 

embedding procedure. The basic hidmg/extracting scheme is jing. We found our image embedding method to be robust, 
similar to the our previous data hiding/extracting technique concluded that it could be easily extended to video 

using the multidimensional lattice structures described watermarking as well 

above and shown in FIG. 7. A single level of discrete cm 11 rym 1^ u *u • 1 •* r *u 

I , , P /r^^rxA f u *u *u u . j *i_ FIG' and FIG. 14 show the similarity of the recon- 

wavelet transformation (DWl) of both the host and the ^.^^^^^ ^ ^^ ^. .J ^^^.^^^ 

signature image is made before data embedding Each « kvels of JPEG compression. A normalized shnilarity func 

coefficient of the signature image is quantized into 6 levels. q/ ^ • j^^^^j „^ 

T J * u J ^- J • . • f . "on S(s) IS denned as 

In order to embed the quanUzed coefficient inform aUon, a set 

of N coefficients in the host image is grouped to form an 

N-dimensional vector, and the vector is then perturbed s{s) ^ 

according to a p-ary channel code consisting of a subset of (^Wi) 

the lattice scaled by a factor a. If v represents a vector of 

host DWT coefficients after grouping, and the index of the where s is the signature image components organized as a 

quantized signature coefficient is i, then the perturbed vector vector, and S is the reconstructed signature vector. As can be 

is given by Equation (5). seen from the graphs, the watermarked image can be easily 
In signature recovery, the watermarked DWT coefficients S5 authenticated even at 85% lossy JPEG compression. FIG. 14 

are grouped based on the P-ary channel code used in shows Peak Signal to Noise Ratio (PSNR) of the recon- 

J , . , . . • 1 J t_ slructed image as a function of JPEG compression factor, 

encoding to obtain a new vector e . This is then scaled by t, dcktd * * j %u • • 1 • 

.u <• ♦ , i/« u • ^ « ^ • n .• /c\ -m. The PSNR IS computed with respect to the original signature 

the factor 1/a where is as defined in Equation (5). The k^f™ w , ^ X * a 1% . 

u , , ■ , • . . J J . c J before quantization. We noted that good quality reconstruc- 

resultant vector is then nearest-neighbor encoded to find the ^„ ^ .,1 * u . -irm mn^ • r 

index i of the channel code nearest to it in the Euchdean '^J"" "P '° compression for 

dfetance. In particular, we find an index i such that Equation ^'^^ y^.^^ Reconstruction without Host Image 
(o) holds true. -n. c t. j- j ■ * 1 

c" i . L c ^u- • -11 . . J • i-i^ A rhus tar, we have discussed image reconstruction where 

Similar to before, this is illustrated in FIG. 12. Assume „ . ■ • -i ui u u *u u . • 

*u«» o.,«,u^i * u » u c host image is available. However, when the host image 

that the symbol s,. was sent but because of compression or • .^umI , 1 • 1 j a u 

. ' u J . 65 IS unavailable, additional complexities are involved. A sche- 

some other image processmg operation, the observed vector An ^..J a . — u ^j- *u -1 r 

_^ matic 40 our data embedding method for reconstruction 

(equal to 1/a e) is obtained. If is within the without the host image is shown in FIG. 15. A key compo- 
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nent of this method is embedding usiag mxiltidimensional 
lattices as previously described. Signature and host images 
arc transformed using the block Discrete Cosine Transform 
(DCI). The block size chosen is 8x8 pixels. The signature 
coefficients arc quantized in two steps. First, by using the 
standard JPEG quantization matrix, and then by a user 
specified signature quantization matrix. The signature quan- 
tization matrix determines the relative size of signature data 
compare to the host data, thus controlling the quantity and 
quality of the embedded data as described in Section 5.1. 
These quantized signature coefficients are then encoded 
using the multidimensional lattices and inserted into the host 
DCT coefficients. This insertion is adaptive to the local 
texture content of the host image blodcs and controlled by 
the block texture factor as described in Section 5.2. The 
steps in embedding are summarized in Section 5.3. 

5.1 Signature Image Quantization 

There is clearly a trade-off between data embedding 
quantity and quality of reconstruction. We method discussed 
below provides a simple scheme here for quantizing signa- 
ture image data using the block DCT quantization matrix. 
This approach enables robust recovery of signature data 
when the embedded image is subject to JPEG compression. 

Consider an 8x8 DCT coefficient matrix. From image 
compression and information theory, it is well known that 
low frequency coefficients require more bits than the high 
frequency ones. One such quantization matrix indicating the 
number of quantization levels for each of the sixty-four 
coefficients is shown in FIG. 16, These quantized coeffi- 
cients are embedded in a lattice structure as described in the 
previous section. For simplicity, we will consider only those 
shells in the lattice stmcture whose elements are {±1, 0}. 
One way of distributing these coefficients is as follows: 

5.1.1 Quantization Levelal232. Use Lattice type E^: The 
first and second shells of lattice combined have 2400 
code words; however, here we tise 1232 code words 
from the combination of first shell and part of second 
shell in this lattice. Since an Eg code has eight 
components, it requires 8 host coefficients to embed 
one Eg code. There are 3 coefficients with this 
quantization, requiring 24 host coefficients to embed. 

5.1.2 Quantization L6vcl='342. Use Lattice type E^: The 
first and second shells of Eg contain 342 code words. 
Six host coefficients arc needed to embed an Eg code. 
The six coefficients in the DCT matrix thus need 36 
host image coefficients to embed. 

5.1.3 Quantization Level=48. Use Lattice type D4: The 
first two shells of D4 are used to encode 48 levels. Each 
code requires four host coefficients. There are thirteen 
coefficients with this quantization, thus requiring 52 
host coefficients. 

Thus, method outlined above thus needs a total of 112 host 
coefficients to embed the 64 DCT coefficients from the 
signature image. 

The next step in embedding is to identify the host coef- 
ficients which are affected by the data embedding procedure. 
The low frequency components contain most of the host 
signal energy but they can not be easily modified as such 
changes may become visible. The high frequency 
components, which usually pack the least amount of energy, 
could be easily removed because of signal processing opera- 
tions. This leaves us with the mid frequency components. 

Consider an 8x8 block of host image coefficients, as 
shown in FIG. 17. The shaded regions indicate the frequency 
components that are identified for encoding the signature 
image data. In this example, 28 host coefficients are used in 
each block, thus requiring four host DCT blocks to encode 
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one signature bbdc It will be appreciated that four host 
DCT blocks (4x28-112) are needed to embed one 8x8 
signamre DCT block. 

Another example of signature image quantization and the 
corresponding host coefficient allocation are shown in FIG. 
18 and HG, 19. Note that 192 host coefficients arc needed 
for this case (6x for Eg, 16x for Eg, and 12x for D4 
=6x8+16x6+12x4=192). One possible way of distributing 
this is shown in FIG. 19 where 12 host coefficients are 
identified for insertion. This requires a total of 16 host DCT 
blocks per signature block. 

5,2 Texture Masking 

The signature coefficients are adaptively embedded into 
the host image coefficients. Recall that insertions into host 
image regions with low texture information would result in 
visible distortions. The texture block factor y controls the 
weighting of the signature coefficients for each 8x8 DCT 
host image block. We use a normalized measure of texture 
energy, defined as: 
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where /*w(B) is the average energy in band B (B«{LH, HL, 
HH}) after a one level discrete wavelet decomposition of the 
host image /foCB) and is the average energy in band of a 
given 8x8 host image block. The term fij(B) characterizes 
the given block texture energy for a given band B. A Haar 
wavelet transform was used in our experiments. If ^B) 
exceeds a given threshold, say T^/B), then the correspond- 
ing block is considered to have significant texture in band B. 
If the block texture energy exceeds the threshold for two out 
of three bands, then the block is considered to be highly 
textured. Similarly, if two out of three band energies fall 
below the threshold T^(B), then the corresponding block is 
considered to be low in texture. 

Each host image DCT block is thus classified into one of 
highly textured, normal, or low textured block, and the 
texture block factor is appropriately set. In the example 
discussed below the following parameter values are used: 

5.3 Data Embedding 

We can now summarize the various steps in the embed- 
ding procedure. FIG. 20 provides a schematic of the encoder 
block 42 of FIG, 15 to show the encoding steps. 

5.3.1 The host and signature images are transformed to the 
DCT domain. A block size of 8x8 is used in the 
example given below. 

5.3.2 Each block of 8x8 host image pixels is analyzed for 
its texture content and the corresponding texture block 
factor y is computed. 

5.3.3 The signature coefficients are quantized according to 
the signature quantization maU-ix and the resulting 
quantized coefficients are encoded using lattice codes. 
The lattice codes are so chosen that the code vectors 
contain only ±1 or zeros. 

5.4 The signature codes are then appropriately scaled 
using the total scale factor 8«»a+Y and the commonly 
used JPEG quantization matrix. The JPEG quantization 
matrix helps in rcnormalizing the code vectors so that 
they have a similar dynamic range as a typical DCT 
block. Note that 6^0, which in turn constraints the 
choice of a and y, 

5.5 The selected host coefficients are then replaced by the 
scaled signature codes and combined with the original 
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(unaltered) DCT coefficients to form a fused block of in a deterministic fashion before distribution. As a result of 

DCr coefficients. Note than more than one host coef- embedding, a mean-squared-error MSE^ is introduced into 

ficient is needed to encode a single signature code. the embedded host. To ensure transparency of embedding, 

5.6 The fused coefiScients are then inverse transformed to MSE„ vahie should be below a certain desired level, 

give an embedded image. As discussed earlier, the s Whde in watermatkmg the allowable MSE„ is very small, 

choice of signature quantizaUon matrix afifects the of signature data. In data hiding the 

J i\ r *!. i_ jj J J * ' c focus IS more on hiding larger amounts of signature data at 

quantity and quality of the embedded data. Choice of ^ ^ ^^^^^ ^^^^^^^ distribution, 

Oie scale parameter a depends on the application. A ^^^^ ^^^ergoes compression and other standard 

larger value a for results m a more robust embedding transformations, llie extraction process may or may not, 

at the cost of quahty of the embedded miage, i.e., there 10 depending on the nature of the application, require knowl- 

could be perceivable distortions in the embedded ^^^^ of original host, to estimate the hidden signature 

image. Asmaller a may result in poor quality recovered the "noisy" embedded host that is received. After 

signature when there is a significant compression of the extraction, it is desired that the channel mean-squared-error 

embedded image. MSEs between the original signature and the extracted 

15 signature be as low as possible. 

EXAMPLE 4 From the discussion of data hiding techniques so far, it 

We used two different sizes for the host image. For will be appreciated that the above dual problems of data 

embedding using the signature quantization matrix of FIG. watermarking, readily map to the source and 

16 and FIG. 17, a 256x256 host image was used, resulting channel coding problem in digital communications. As such, 

in 25% data embedding. A 512x512 host image was used ^ estabhshed concepts from digital communications could be 

with the quantization matrix of FIG. 18 and FIG. 19. used to solve this problem ^ ^ ^ ^. 

, , , t., d1 IJ^^^ Hidmg using Vector Perturbations 

We examined ^6 embedded images with and without According to the present invention, the host data is 

texture maskmg. The signature quantizaUon matrix shown in orthogonaUy transformed before embedding the hidden sig- 

FIG. 18 and FIG. 19 was used for this purpose. We found ^^^^^^ y^e transform is not essential because a raw 

that texture masking reduces visible distortions in regions ^^^^ ^-^^^ ^^^^ ^„ expansion on the standard 

that are flat. bases. However, it may lead to some advantages. Let us 

We also examined recovered host and signature images consider a host data source (X^, X2, . . . , X^) transformed 

for two different quantizations of the signature data, using orthogonally to a set of N coefficients (Cj, Cj. C^,^). The 

texture masking. In this case, the embedded images were 30 transform-domain embedding process perturbs the coeffi- 

lossy compressed by JPEG to 89%. Obviously, the quanti- cients into a new set of coefficients given by (Cj, . . . , 

zation matrix of FIG. 18 and FIG. 19 yields better results C^). xhe inverse transformation then yields the embedded 

than the one shown in FIG. 16 and FIG. 17 at the cost of host (Xj, X^, . . . ^j^)- Since the transformation is 

more host bits per signature coefficient. orthogonal, the mean-squared-error introduced in the coef- 

Finally, FIG. 21 and FIG. 22 show the quality of the 35 ficients is exactly equal to the mean-squared-error intro- 

embedded and recovered images using the PSNR as a duced in the host data. That is, 
measure. It is clear from these graphs that one can achieve 

better quality embedding using the quantization matrix of ^ n t i ^ a ^^^^ 

FIG. 18 and FIG. 19 at the cost of lower bit rate for the MSEh = ^7 -^1'^' " ^^'l = /v ' Zl^' " ^'1 

hidden data. We found that even at 25% embedding, one can 40 '"^ '^^ 
recover visually acceptable quality results for up to 90% 

lossy compression using JPEG. nq^, a transparency constraint is imposed on the value of 

It will be seen, therefore, that the invention provides a MSE^^. This specifies a maximum value P which upper 

robust data hiding technique for embedding images in bounds MSE^ for a given appHcation: 

images. A key component of the scheme is the use of 45 

multidimensional lattice codes for encoding signature image ^ n 2 1 ^ 

coefficients before inserting them into the host image DCT '^^^ n '2 I^' ~ ^'I 

coefficients. Texture masking is used to reduce distortions in '"^ 
the embedded image by adaptively controlling the weights 

associated with the hidden data. The hidden signature data so jhe smaller the value of P, the more transparent the embed- 

can be recovered in the absence of the original host image. ^j^^ vice- versa. 

Experimental results show that this method is robust to lossy sij^^e N is typically very large for images and video, it 

image compression using JPEG. One can trade-off quantity ^^^^^^ ^^^^ to simplify the transparency constraint by 

for quahty of the embedded image by choosing appropriate grouping the N coefficients into k-dimensional vectors with 

signature quantization matrices. 55 ^^^^^ ^^d satisfying the constraint in each of the vectors 

6. Hidmg Speech m Video individually. Further, it may be necessary to perturb only a 

In order to hide speech in video in accordance with the hmited number M of the N coefficients, say the coefficients 

present invention, the host video is wavelet transformed Q^ly one particular band of a subband or wavelet decom- 

frame by frame, and vectors of coefficients are perturbed position. That is, if the M coefficients to be perturbed are 

using lattice channel codes to represent hidden vector quan- 60 grouped into M/k vectors of dimension k, denoted as VJ-1, 

tized speech. The embedded video is subjected to H.263 2, . . . , M/k, and the corresponding perturbed vectors are 

compression before retrievmg the hidden speech from it. denoted as V,, then for each of the vectors, the following 

The retrieved speech is intelligible even with large com- nj^st be satisfied to satisfy the constraint in Equation (12): 
pression ratios of the host video. 

FIG. 23 presents a basic schemadc 50 of the data hiding 65 Vh\\Vj.Vf<P^-K/M-Pj^ix .--.Mlk (13) 

and watermarking problem as it applies to hiding speech in At this stage we can explain the general embedding 

video. The original host is modified using the signature data principle by means of the diagram in FIG. 24. The signature 
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data is first coded, either losslessly or lossily, to generate a 
sequence of symbols from a Q-ary alphabet {sj, s^, . , . , s^}. 
The embedding process injects one symbol in each coeflB- 
cieot vector Vy , by perturbing it in one of Q possible ways 
in k-dimensional space to obtain the perturbed vector . 
Note that Ihc possible values of Vy all lie within a shell of 
radius VkP^ from V^, to satisfy the transparency constraint. 
The possible perturbations constitute what is in general 
known as the channel codebook, of size Q and dimension k. 
The channel codebook is usually obtained from a noise- 
resilient channel code by scaling it by a factor a which 
determines the transparency constraint. That is, the per- 
turbed vectors are obtained as: 

PrVf^C{s,l (14) 

where the set of vectors C(s,), i^l, 2, . . . , Q constitute a 
channel shape codebook of size Q. The perturbed coeffi- 
cients are used to inverse transform the host before trans- 
mission or distribution. 

The extraction principle is outlined in FIG. 25. Let us say 
that the jth distributed perturbed vector Vy, corresponding to 
a symbol s,., has been received as Wy, as a resuU of an 
additive noise ny due to compression and other transforma- 
tions. However, as long as the received vector does not go 
beyond certain pre-dctcrmincd decision botmdaries for sym- 
bol s^, the correct transmitted symbol s^ will still be 
extracted, provided the true original host is known. The 
recovery process thus extracts from each vector the symbol 
within whose decision boundaries the received vector lies. 
In other words, a nearest neighbor search with an appropri- 
ate distance measure is used. The decision boundaries 
depend on the statistical model chosen for the additive noise. 
The sequence of extracted symbols are then decoded to 
obtain the extracted signature. 

Some comments are now in order. First, we can define a 
rate R for data injection in bits/dimension as follows: 

/!-l/itlog2e (15) 

Next, assuming an i.i.d, additive white Gaussian noise 
(AWGN) model for the pixels in the distributed host, and 
therefore its orthogonal transform coefficients, the extraction 
process becomes a simple nearest-neighbor encoder with the 
Euclidean distance measure and symmctric-hyperplane 
decision boundaries. Moreover, if we assume the AWGN 
variance to be cr^, we can define a channel capacity, C as: 




Thus, P^, obtained by scaling the transparency constraint P 
by a factor (N/M), can be viewed as the power constraint on 
the channel. According to Shannon's celebrated theorem, as 
long as 

R <C, virtually error-free transmission can be achieved by 
choosing a sufficiently large dimension k. The term C is the 
theoretical upper-bound on the error-free rate a AWGN 
channel can sustain for a given power constraint. 
Unfortunately, the upper-bound can only be achieved for 
infinite dimensionality k. In practice, the larger the dimen- 
sion k, the more noise resilient the channel coder is. 
Therefore, the dimensionality of the vectors should be 
increased as much as possible. 

Finally, with increase in the amount of signature data, it 
makes sense to lossily source code the data if it is com- 
pressible. A method that works well for correlated sources is 
vector quantization. The indices obtained by vector quanti- 
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zation are embedded into the host transform coefficients by 
vector perturbations derived from noise-resilient channel 
codes. Note that it is also possible to design channel- 
optimized VQs (COVQ), or Power-Constrained COVQs 

S (PCCOVQ) for better noise performance. 

In the present invention, the channel codes arc chosen as 
subsets of lattices in multiple dimensions. It is known that 
the lattices D4, Eg, Eg, Kjj* Ajg, etc. produce very good 
channel codes in their respective dimensions, and tables and 

10 graphs with their nominal coding gain results arc commonly 
available. 

Most of our implementations are based on spherical or 
constant-energy codes, for which, all the points are equidis- 
tant from the origin. With such codes, the MSE^ introduced 

15 as a result of embedding is exactly equal to the transparency 
constraint. In practice however, for image and video hosts, 
the effect of rounding the pixels of the embedded host to 
integers, and limiting them to he in the range of 0-255, may 
cause minor deviations from the theoretical value. 

20 6.2 Recovery from Video Host without Original 

The general principle of data hiding in video is as follows. 
Each frame of a video sequence is orthogonal wavelet 
transformed, and the transform coefficients are grouped into 
vectors. The signature data is vector quantized, and the 

25 indices are embedded into the coefficient vectors in one or 
more subbands using efficient channel codes. The same 
hidden data may be repeated in a few successive frames to 
introduce robustness to low frame rate compression of 
video. Note that the frame by frame approach fits very well 

30 with the frame -based compression technology currently in 
vogue. 

We now focus on the issue of choice of subband for 
embedding the hidden data. When the original host is 
available during retrieval, and the kind of host transforma- 

35 tion we are most concerned with is compression, hiding data 
in the lower subbands has several distinct advantages. Most 
modem compression schemes quantize the lower bands 
finely, and in some way exploit the fact that the higher bands 
have very little energy. Injecting extraneous information 

40 only in the lower bands, and leaving the higher bands 
untouched, therefore, reduces the probability of destruction 
of the hidden information, and at the same time does not 
affect any significant change in the coding efficiency. 
Although a disadvantage is that the distortions introduced by 

45 embedding may be perceptually more severe, weighing the 
pros and the cons, hiding data in the lower subbands is still 
found to be better. 

If, however, extraction is to be made possible without 
knowledge of the original host, hiding data in the lower 

50 bands is not appropriate. The key idea behind a data hiding 
scheme that allows extraction without the original host, is to 
convert the original host conveniently before embedding to 
a slightly different one, and to use that as the base host for 
embedding, instead of the true original. The modification 

55 introduced must be such that it becomes possible to estimate 
the base perturbed vectors from the received host, with the 
modified base host being only trivially dissimilar to the true 
original. Natural images typically have very low energy in 
the high-high bands. Therefore, a simple zeroing out of one 

60 or more of the high-high bands, introduces a very low MSE, 
and for most images, affects image detail only inconspicu- 
ously in the perceptual sense. If a modified base host is 
obtained by zeroing out one or more of the subbands of the 
original host, the extraction process only needs to use the 

65 zero-vector as the estimation base for the perturbed vectors 
it receives within these subbands. This however, contradicts 
the requirements in the previous paragraph, that it is better 
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to embed data in the lower subbands. To make a 
compromise, the following methodology is adopted. As 
shown in FIG. 26, a two-stage wavelet decomposition of 
each frame is made and the data is hidden in the shaded 
LL-HH subband after zeroing. 5 

It is appropriate to make a comment on the zeroing out 
approach described above. Zeroing out one or more bands 
before embedding may result in significant distortions or 
loss of detail for some host videos. A greater transparency of 
embedding may be achieved if the coefficients in the con- lO 
cemed subbands in the base host are predicted, linearly or 
non-linearly, from the coefficients in the other subbands that 
are not zeroed out. Specifically, if the prediction used is 
linear, and the noise is assumed to be additive i.i.d. 
Gaussian, it can be shown that the noise in the predicted base 15 
coefficients will still be Gatissian. The estimation of the 
transmitted symbols will then be essentially the same prob- 
lem as before, but at a higher noise level. In general 
however, linear prediction across subbands docs not lead to 
any significant advantages. Obtaining the best nonlinear 20 
prediction across subbands, on the other hand, is a very 
difficult problem. Further, this leads to the difficult problem 
of estimation of the base coefficients in the embedded 
subbands, firom the already noisy coefficients in the other 
subbands, at the retrieval end. In this case, the predicted base 25 
coefficients will no longer be Gaussian, and consequently, 
the decision boundaries for extraction may be very complex. 
In this work, we have sidetracked the issues involved by 
adopting a simple zeroing out approach, which works very 
well in practice. 30 

FIG. 27 and FIG. 28 show schematic diagrams 60, 70 for 
the embedding and extraction mechanism outlined above, 
respectively. The host video is first wavelet transformed. An 
encryption key is used to pseudo-randomly shuffle the 
coefficients in the subband chosen for embedding before 35 
grouping them into k-dimensional vectors. The hidden com- 
pressible data is appropriately vector quantized, and the 
indices obtained in the process are embedded into the 
k-dimensional host transform vectors by vector perturba- 
tions in accordance with efficient channel codes scaled by a 40 
factor a. The encryption key based shufEling introduces an 
additional layer of security apart from the security enforced 
by the already astronomic variability in the source and 
channel codebooks chosen. It is virtually impossible for 
unauthorized persons who know the algorithm, to pirate the 45 
hidden information, without knowledge of the source 
codebook, the channel codebook, or the encryption key. 

Another advantage of using pseudo -random shuffling of 
coefficients to form vectors is as follows. Typically, the noise 
introduced as a result of transformations such as compres- 50 
sion in a frame occur in "bursts". That is, a heavily corrupted 
coefficient is likely to have its neighboring coefficients also 
heavily corrupted. Therefore, if adjacent coefficients are 
grouped to form vectors, the noise in the components remain 
too correlated to fit our assumed model of being independent 55 
and identically distributed. Shuffling implies that the com- 
ponents of a vector now come from different random parts 
of a frame, and therefore, the noise introduced in the 
coefficients become closer to being i.i.d. This in turn vali- 
dates the use of the Euclidean distance measure for channel 60 
decoding. 

EXAMPLE 5 

We implemented a system for hiding 8 kHz sampled 
speech at 16 bits/sample in a 30 frames/s QCIF video, 65 
without requiring the availability of the original video for 
retrieval. The speech and video were synchronized in time. 



Successive samples of speech were vector quantized, and the 
indices were embedded into the LmH subband coefficients 
of the video on a frame-by-frame basis. Temporal redun- 
dancy was incorporated by embedding the same information 
in several successive frames, so that the embedding becomes 
robust to frame skips during compression. 

First, we attempted embedding the signature speech in 
only the luminance LL-HH subband. The embedded video 
was piped through a H.263 encoder as before, and the 
reconstructed video is used to extract the hidden speech 
segment. We present the details of three different implemen- 
tations with increasing dimensionality of the channel codes 
used: 

(a) The speech is vector quantized with a codebook of size 
576 and dimension 4. The index obtained was decom- 
posed into two 24-ary symbols, each of which was 
embedded into a vector of dimension 4 obtained by 
grouping four luminance LL-HH coefficients of a two- 
stage wavelet decomposition. The embedding was done 
by perturbing the vectors in accordance with a spherical 
channel code consisting of the first shell of the D4 
lattice (which has 24 points). 

(b) The speech codebook is of size 240 and dimension 4. 
The index for each speech vector was used to perturb 
a group of 8 luminance LL-HH coefficients in accor- 
dance with a spherical channel code comprising the 240 
points on the first shell of the Eg lattice. 

(c) The speech codebook is of size 4320 and dimension 8, 
The encoded index was embedded into a vector of size 
16 obtained by grouping 16 luminance LL-HH coeffi- 
cients. The channel code comprised the 4320 points on 
the first shell of the Barnes- Wall Lattice A^^. 

For all the above implementations, the same information 
was repealed in two successive frames to introduce robust- 
ness to low frame rate compression. The News QCIF video 
was used as the host for hiding a segment of male speech. 
The signal to noise ratio for the extracted speech segment 
against the video bit rate after H.263 compression of the host 
at 15 frames/s (frameskipol) is plotted in FIG, 29. The 
transparency constraint was the same for all these results. As 
expected, the highest dimensional lattice A 15 was found to 
be most robust to noise. 

We next present the results for three implementations 
where both the luminance and the chrominance coefficients 
arc perturbed: 

(a) The speech codebook is of size 5184 and dimension 8. 
Each index was decomposed into two 72-ary symbols, 
which are embedded into two coefficient vectors of 
dimension 6. Each 6-dimensional coefficient vector 
was obtained by grouping 4 lummance LL-HH coeffi- 
cients and 1 LL-HH coefficient from each chrominance 
component. A spherical channel code derived from the 
first shell of the Eg lattice (which also has 72 points) 
was used for each symbol. 

(b) The speech is vector quantized with a codebook of size 
756 and dimension 8. A 12-dimensional coefficient 
vector was obtained by grouping 8 luminance LL-HH 
coefficients and 2 LL-HH coefficients from each 
chrominance component, A spherical channel code 
consisting of the 756 points on the first shell of the 
Coxeter-Todd lattice K^j was used. 
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(c) The speech is vector quantized with a codebook of size 
4096 and dimension 16. A 24-diraensional coefficient 
vector was obtained by grouping 16 luminance LL-HH 
coefficients and 4 LL-HH coefficients from each 
chrominance component. A spherical channel code 
G24, consisting of 4096 points, was used. G24 was 
obtained from the (24, 12) extended Golay code by 
converting zeroes to ones, and ones to negative ones. 
For all the above implementations, the same information 
was repeated in four successive frames. FIG. 30 presents the 
retrieval SNfR vs. bit rate results for the above methods when 
a segment of female speech was hidden in a Grandmother 
QCIF video, which was then coded by H.263 at 7.5 frames/s 
(frameskip-3). The transparency constraint was the same for 
all these results. As expected, the highest dimensional lattice 
G24 was found to be most robust to noise. 

As can be seen therefore, the foregoing provides a generic 
framework for hiding compressible data in host video. Our 
MSE-optimal quantitative treatment is motivated by the 
identification of the similarity of the data hiding problem 
with the source and channel coding problem in digital 
communications. While the generic approach can be used 
successfully for the case when the original host is available 
to the retriever, the true potential of data hiding lies in being 
able to extract the hidden data without using the original 
host. The above method is readily adapted to allow this, 
making possible invisible mixing of different kinds of hid- 
den data, with standard forms of open data transmission. 

Although the description above contains many 
specificities, these should not be construed as limiting the 
scope of the invention but as merely providing illustrations 
of some of the presently preferred embodiments of this 
invention. Thus the scope of this invention should be deter- 
mined by the appended claims and their legal equivalents. 

TABLE 1 



Code Types and structure of the D4 lattices 



Shell No. Squared Norm Sourre codes Number of codes 



1 


2 


(ti. ti, 0, oy 


24 


2 


4 


(a:2, 0, 0, 0)^ 


24 










3 


6 


(±2, ±1, ±1, 0)P 


96 


4 


8 


(tZ, £2, 0, 0)'* 


24 


5 


10 


(t2, ±2, ±1, ±iy, 


144 






(r3, il, 0, 0/ 





TABLE 2 



Quantizer Level (D^ lattice') 
Quantizer Levels p Lattice points in channel code 



2 (0, 0, 1, 1), (0, 0, -1, -1) 

24 Shell J 

32 ShcUi, (t2, 0, 0, Of 



1,030 Bl 

24 



TABLE 2-continued 



5 



Quantizer Level (D^ latticed 


Quantizer Levels ^ 


Lattice points in channel code 


48 


ShclU, Shclla 


144 


Sbclli. Shclla Shells 


168 


Shclli, Shclla Sheila, Shelly 



10 

What is claimed is: 

L A method for embedding a signature image in a host 
image, comprising: 



15 (^) performing a single level discrete wavelet transform 
decomposition of said signature image and said host 
image; 

(b) quantizing into p levels each coefficient of said 
signature image by grouping a set of n coefficients in 

20 the host image to form an n-dimensional vector, and 
perturbing said vector according to a p-ary channel 
code comprising a subset of an n-dimensional lattice 
scaled by a factor a; 

^ (c) embedding each subband of said signature image into 
a corresponding subband of said host image to produce 
a composite image; 

(d) subtracting the coefficients of said host image from the 
coefficients of the composite image to obtain noisy 

30 perturbations; 

(e) grouping the resulting coefBcients into groups of n to 
obtain a vector e ; 

(f) scaling said vector e by 1/a to produce a resulting 
vector l/a T; 

(g) nearest- neighbor encoding 1/a- e to find an index i of 
the channel code nearest to it in Euclidean distance; 

(h) obtaining a quantized discrete wavelet transform coef- 
ficients from said index i. 

2. A method for embedding an audio signature in a host 
video image, comprising: 

(a) encoding said audio signature to generate a sequence 
of symbols from a Q-ary alphabet {Sj, Sj, . . . , s^}; 

(b) injecting one symbol in each coefficient vector Vy, by 
perturbing it in at least one of Q possible ways in 
k-dimensional space to obtain the perturbed vector V^-; 
and 

50 (c) using perturbed coefficients to inverse transform said 
host video image and produce a composite signal. 

3. A method as recited in claim 2, further comprising 
extracting from each perturbed vector the symbol within 
whose decision boundaries the vector of the composite 

55 signal hes. 

t lit * * * 
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